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Abstract. Understanding Research and Development (R&D) enterprise relationships and processes at a governance level is not a 
simple task, but valuable decision-making insight and evaluation capabilities can be gained from their exploration through computer 
simulations. This paper discusses current Modeling and Simulation (M&S) methods, addressing their applicability to R&D enterprise 
governance. Specifically, the authors analyze advantages and disadvantages of the four methodologies used most often by M&S 
practitioners: System Dynamics (SD), Discrete Event Simulation (DES), Agent Based Modeling (ABM), and formal Analytic Methods 
(AM) for modeling systems at the governance level. Moreover, the paper describes nesting models using a multi-method approach. 
Guidance is provided to those seeking to employ modeling techniques in an R&D enterprise for the purposes of understanding 
enterprise governance. Further, an example is modeled and explored for potential insight. The paper concludes with 
recommendations regarding opportunities for concentration of future work in modeling and simulating R&D governance relationships 
and processes. 


1.0 INTRODUCTION TO RESEARCH 
AND DEVELOPMENT GOVERNANCE 

Key tasks of governance depend on many 
considerations, but potential governance- 
level responsibilities may include strategy 
and operational framework design, vision or 
direction of organization propagation, goal 
setting, funding allocation, risk reduction 
and uncertainty assessment, effectiveness 
and performance assessment, decision 
making regarding management or lower 
level structures, policy adjustments, and 
policy adherence verification. 

Governance of Research and Development 
(R&D) is a very broad term and has different 
implications for public and private sectors, 
however, some rules and perspectives are 
quite similar. The goals and objectives of 
these two groups are quite different, 
however both provide a frameworks that 
encourages desirable research outcomes 
for enhanced mission capabilities [1], Here, 
it is important to understand major 
differences regarding how both sectors see 


and evaluate their mission capabilities. 
Private sector governance concentrates on 
investments, product lifetime cycle, future 
planning, and sustainability [2], but the 
primary goal of many private enterprises is 
to make money. Exceptions from this rule 
include non-profit organizations or 
corporations where effectiveness aligns 
closer to public sector measures. Public 
sector governance focuses on different 
outcomes such as leadership, policy 
soundness, and innovation creation, 
elements traditionally viewed as specific 
outputs from more detailed managed 
endeavors. Money spent is an important 
factor, but the main organizational 
objectives do not typically focus on a return 
on investment paradigm. 

Another context to consider is the level of 
R&D governance, because it may span 
across a society, country, industry, 
company, or individual endeavor within an 
organization. Different approaches to R&D 
governance can be more effective with 
different organization sizes and 
complexities. Another factor to study is the 
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composition of different R&D organizational 
structures, for example: centralized, de- 
centralized, or mixed; this can have a 
significant impact on performance of the 
organization [3], Assessment of an R&D 
endeavor is tightly coupled with 
organizational goals. Input in the form of 
goal setting will direct the R&D process 
through execution of projects, and ultimately 
it will feedback to the governing cell in the 
form of, for example, higher profits, 
increased market share in the private 
sector, lowered crime rates, decreased 
cancer-related deaths, and an increase of 
the approval rating of a given organization 
in the public sector. A formulated 
governance strategy establishes a context 
for R&D entities and a consistent basis for 
funding requests, adjustment of programs or 
possible outsourcing to accomplish 
organizational goals. It is also an enabler of 
generating new ideas for future projects, 
which would help to fill the gap between 
current state, and desired organizational 
objectives. 

The remainder of this paper is organized as 
follows. Section 2 discusses the applicability 
of M&S techniques to R&D governance. 
Section 3 briefly presents the most common 
M&S paradigms. Section 4 discusses the 
problem of choosing a modeling paradigm 
in the context of R&D governance. Section 

5 presents a sample model. Finally, Section 

6 provides some conclusions regarding this 
research. 

2.0 NEEDS FOR MODELING IN R&D 
GOVERNANCE 

Decision-making in complex systems 
without sufficient information and 
understanding could lead to catastrophic 
results. Further understanding of complex 
systems may be achieved through M&S 
techniques. Frequently, available data do 
not translate into valuable information that 
facilitates understanding. When investing a 
significant amount in a research project, one 
should first know that a proposed solution 


has a high chance of success, and if the 
proposed solution is actually addressing the 
concern [4], Another problem can be 
discovered while looking at reports of 
government agencies where many 
voluminous reports do not provide clear 
information on the bigger picture context. 

Measures of “target met”-type results for 
particular research endeavors do not 
provide a sufficient way to validate a 
decision-making process and its evaluation. 
On the other hand, this situation is likely due 
to a lack of knowledge on how to 
successfully describe R&D processes, and 
their relations at a sufficient level of 
abstraction. The problem can be 
instantiated into the scope of conceptual 
understanding to see how this system 
works. If one knew how the system 
behaves, a model could be built out of this 
knowledge to aid in decision-making 
process. This sounds straightforward, but in 
fact, it is not. There are many parts of the 
system influencing each other and it is 
difficult to describe these relations in a 
mathematical formalism. Further, human-in- 
the-loop concerns are still a big challenge to 
model efficiently, and when dealing with 
innovation it is obvious that not only 
qualifications of scientific staff, but also 
other factors such as their motivation, 
character, and needs, which could influence 
the culture and performance of the working 
group are important. Therefore, evaluation 
of the process of innovation generation can 
be subjective and a decision-maker 
concerned with organizational governance 
should be aware of that. M&S practitioners 
are working diligently trying to fill this gap, 
because common analytical techniques 
cannot. 

Agent Based Modeling is one of the most 
promising techniques that could generate 
an understanding of innovation 
phenomenon and emerging group 
behaviors. A question often asked is how to 
asses and measure R&D effectiveness. 
One way of approaching the problem is to 
focus on internal and external factors that 
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facilitate effectiveness and not on the 
innovation product itself, especially when 
assessing a long-term project, where it is 
difficult to gauge the progress of research. 
Another way is to find variables directly 
coupled with outcomes from an R&D 
endeavor, keeping in mind that some 
variables can be measured, but some can 
only be observed as relative trends. Length 
and difficulty of the R&D endeavor can have 
a significant impact on accuracy of the 
prediction, therefore overall assessment as 
well. When considering the length it is 
apparent that long-term R&D endeavors 
should be more difficult to appraise and the 
probability that one will conduct an 
erroneous assessment is higher. By 
employing modeling techniques, one could 
address some of the problems mentioned 
here. Modeling the innovation process and 
its supporting activities can be achieved 
using modeling techniques. They can 
alleviate a problem of R&D effectiveness 
assessment and help to evaluate trends and 
conduct “what if” analysis on different styles 
of decision-making and different polices of 
R&D governance that oversee innovation 
generation. Successful conclusion of R&D 
innovation generation is not the end of the 
lifecycle process. Another responsibility of 
R&D governance is to incorporate a new 
product or concept into a market or its 
functional target place. This task can be 
very complex and traditional methods may 
not suffice. M&S can mimic time-related 
variables and environmental relationships 
related to new inventions to optimize the 
process of integration to maximize 
outcomes, and to avoid pitfalls that could 
possibly occur. 

Another area where M&S can be invaluable 
is assessment of future R&D direction. 
Conducting analysis of R&D effectiveness 
using modeling techniques in the context of 
organizational mission can reveal serious 
strategy flaws and could help to avoid them. 
M&S methods are relatively new 
approaches, as a choice for aid in decision- 
making and evaluation of R&D governance. 
Authors were looking into areas where M&S 


is already more advanced to analyze 
options in applying robust knowledge to 
R&D governance. 

The most prevalent area of M&S is 
associated with, and sponsored by, military 
organizations. In [5], authors used ABM to 
model Political, Military, Economic, Social, 
Information and Infrastructure (PMESII) 
entities to mitigate the impact of disasters 
and military operations. This model has 
been used to represent the rebuilding and 
expansion of critical infrastructure in 
Afghanistan. The ability to gain insight into 
the process of reconstruction and 
development can be a very valuable tool for 
a decision-maker in enterprise or 
government structures. Because of the 
multi-dimensional scheme of the model, the 
authors believe that it would be possible to 
apply this approach to R&D governance. 

Adding modules that would allow agents to 
model innovation generation and knowledge 
sharing is possible with modeling by 
connecting this lower level to middle and 
higher-level agents, e.g. department, 
organization, and society. A model could 
allow the analysis of the robustness of 
policy choices under different scenarios. 
This approach can be used in the public as 
well as private sector. Adding more 
complexity to the model would theoretically 
allow for modeling human evolution as a 
function of R&D progress. Because the 
progress of R&D could be an enabler for 
actual change of the system and can spur 
achievements of higher objectives of system 
of systems, it can be invaluable to be able 
to model this process. As a futuristic 
perspective, such a model can provide 
valuable information on R&D as a stabilizer 
or destabilizer of the world, and perhaps 
allow answering the following questions: 

Is advancement of R&D proportional with 
civilization advancement? 

Is our direction of R&D advancement taking 
us where we want to be? 

Because of its complexity, currently only 
large organizations can handle expenses 
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related to investing in complicated models 
conveying many levels of abstraction, but 
this can change as the field of M&S 
advances, providing for easier model reuse 
and better interoperability. The following 
section discusses the most common 
modeling paradigms to familiarize the 
reader with what approaches are available. 

3.0 MODELING PARADIGMS 

In this section, the authors would like to 
provide quick overview of three popular 
and often-used-by-practitioner M&S 
methodologies. Understanding principles of 
them is an important step in understanding 
their possible usage, as discussed in the 
next section. 

3.1 System Dynamics 

System dynamics (SD) models consist of 
feedback loops in the form of differential 
equations that provide for building relations 
between variables. SD is useful for studying 
complex nonlinear systems, especially 
finding cause and effect relationships [6], 
From the practical perspective on how to 
build model, one should know about stocks, 
flows, auxiliary variables, and constants as 
the main blocks providing for metaphors of 
the complex system. The initial phase of 
building SD model is creating a causal 
diagram. A causal diagram represents the 
most important elements of a system and 
relations within the system in form of links 
that end with arrows indicating which 
element influences which. In addition, you 
can denote a “+” or on the flow to specify 
a positive or negative relationship. The 
positive one means that second variable 
follows the direction of change of the first 
variable, and vice versa. Links with 
variables form loops, and they can be 
positive or negative as well. The negative 
loop can exist only if the number of negative 
relationships is odd. Having at least one 
negative loop is a minimum requirement to 
stabilize the system. Models built with this 
methodology can help in framing problems, 
revealing dynamics related to change 


imposed on the system. Models are typically 
used to show trends of relationships and not 
precisely computing specific values. This 
methodology can be an invaluable tool in 
assessing a big picture problem, testing 
alternative policies and strategies at the 
governance or enterprise level. 

3.2 Agent-Based Modeling 

The world is full of interactions between 
different types of systems and many of them 
are not well understood. Agent-based 
modeling (ABM) is a methodology that aims 
to capture many types of interactions by 
using computer created entities called 
“agents”. These agents are assigned 
attributes, rules of behaviors, mimicking and 
carrying sets of interactions to gain an 
understanding of the real system, and 
providing for emerging behaviors. ABM can 
be used to model perception, autonomous 
and group behaviors, goal setting, a static 
or dynamic environment, the ability to mimic 
inference and adaptation, and social 

interactions among other possibilities [7], 
ABM in R&D governance allows for the 
examination of different types of relations 
between individuals, groups, and 
organizations, with the notion of 

independence of agents, when modeling for 
example open innovation or supplier- 

customer relations. These concepts clearly 
could take advantage of ABM to reveal 
some of their intricacies. For these 

reasons, it is popular to use ABM to 
generate hypotheses about system 
behavior. From the scientific point of view, 
this is a very powerful feature, but there are 
some practical limitations related to an 
inability to logically validate such models. 
Consequently, verification and validation 
(V&V) is an issue that needs to be resolved 
to take full advantage of ABM. One way of 
partially alleviating this drawback is by 
proving a hypothesis false. 

Complexity and dynamic behavior of 
systems encourages the use of ABM for 
investigation of adaptive systems, which are 
often non-linear and chaotic. Furthermore, 
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provided mechanisms lead to model 
evolution of adapting agents [8], This 
capability sets the ABM methodology apart 
from other methodologies discussed in this 
paper. It is worth mentioning that relatively 
high complexity can be achieved by 
imposing relatively simple interaction rules 
on agents. There are many platforms for 
implementing AMB, e.g. NetLogo, Swarm, 
Mason, Repast, and AnyLogic. The choice 
of one of these platforms would greatly 
depend on the complexity of system that 
one would like to model and the problem 
that the constructed model is supposed to 
address. 

3.3 Discrete Event Simulation 

In a discrete model, changes in a virtual 
system can occur only at separate points in 
time. These changes are called events, and 
everything in the model is related to them in 
one way or another. For example, an event 
could be an arrival of a R&D project for its 
approval, then decision made would mark 
next event, following with outcome event 
from the project e.g. success or failure. 
Discrete Event Simulation (DES) model 
consists of entities, which in an R&D 
environment could be represented by R&D 
projects flowing through model logic. Events 
are stored in a calendar, which contain 
information that allows the model to be 
executed in accordance with its logic. The 
central idea of DES is that variables of the 
model will not change between successive 
events. 

In addition, understanding the concept of a 
queue is often necessary to the 

understanding of DES. A queue can be 
visualized like a line in a store, and can be 
modeled to mimic the system’s real queue, 
which may have limitations for the number 
of elements that can fit into it, and can have 
different rules reflecting the priority of 
leaving it by the stored elements, e.g. a first 
in first out relationship. 

Essential components of DES are 
represented by resources, which relate to 


personnel, equipment, etc. of the real 
system. They will be used by entities while 
going through the process. DES is often 
used to capture stochastic behavior of a 
system [9, 10], 

4.0 MODEL CHOICE 

One can model R&D governance at various 
levels of abstraction, complexity, and fidelity 
depending on purpose of a model [11]. It is 
important to understand some basic 
limitations of M&S to proceed with this 
discussion further. One needs to remember 
that modeling activity is not free, but can, if 
executed properly, save a great deal of 
money and time. 

Model development and validation often 
requires a significant amount of time and 
resources. No one model answers all 
questions, so it is important to evaluate cost 
effectiveness when deciding to use M&S 
techniques, meaning, if one can address a 
problem with simple deterministic and 
analytical calculations, then M&S is most 
likely not needed. Analytical Methods (AM) 
are usually quicker and cheaper. AM are 
often used to get an exact answer to a 
problem, usually where there is a direct and 
known relationship between input and 
output. 

Short length term R&D endeavors should be 
especially good candidates for use of AM in 
effectiveness assessment at the managerial 
level. This could be done, for example, by 
combining R&D into business process. In 
some cases, AM allow for quick estimates 
for a problem, which then needs to be 
addressed more thoroughly with a M&S 
paradigm, as AM would not allow to capture 
the bigger scope or an appropriately 
complex problem. 

Generally, the choice of a modeling 
technique depends on what features of a 
system can be effectively manipulated with 
different methods. Aggregation level is 
another factor that will influence the choice 
of modeling technique. It is important to 
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understand that a lower level of aggregation 
does not necessarily translate into lower risk 
associated with, e.g., prediction. At the 
lower level of abstraction of R&D 
governance one can focus on innovation 
generation, knowledge management, 
process of innovation control and 
supervision, and project management 
scope. These could provide an instrument 
used for medium and higher level 
structures, or can be studied on their own 
as separate models. Modeling emerging 
behavior using ABM could potentially be a 
method to use at the lowest level of 
abstraction of R&D governance. One could 
get insight into new order, new patterns, 
and structures of R&D by learning from the 
interaction of individual agents or groups. 

One possible approach to implementation 
would consist of distributed agents in the 
form of humans, information storage, or 
groups working together, with attributes 
such as goals, knowledge, and imposing 
adaptive tension on one another [12]. 
Outcomes from the emerging behavior can 
be greater than the sum of its components, 
e.g. knowledge generation; tacit knowledge 
depends on interaction and direct 
experience, whereas explicit knowledge 
could easily be transferred across distance. 
Modeling a spectrum of interactions would 
allow for a better understanding of complex 
emerging behaviors, along with 
quantification of expected knowledge 
increase. Different structures at different 
levels in enterprise or government along 
with imposed interactions could, for 
example, increase chances for innovation 
generation. Emerging behavior capabilities 
of ABM can provide for building new 
theories, which in turn can be translated into 
better structures, encouraging motivation, 
and knowledge sharing in an organization 
that enables creativity. As discussed, it is 
feasible that ABM can be mostly suitable for 
theoretical development, which should be 
later validated using methods that are more 
empirical, as ABM is generally difficult to 
validate. ABM can also be suitable at higher 
levels of R&D governance modeling, where 


individual agents mimic projects, 
organizations, societies, regions, and 
countries, and agents' independence can 
provide for different perspectives comparing 
to one obtained from, e.g., the SD method. 

DES seems appropriate to use when 
looking at R&D governance from the 
processing perspective [13], Every big 
enterprise or government entity has to 
manage many R&D projects, evaluate them, 
and properly allocate resources. DES allows 
for modeling of R&D endeavors throughout 
their stages in a life cycle. In DES, R&D 
projects can be represented as entities 
going through their life cycle, starting from 
project approval request, which can end up 
as a rejection, then the development phase 
through stages at which it is possible for a 
project to be rejected from continuation due 
to, e.g. a lack of funding. This would allow 
the transferring of resources allocated to 
this project to another project, increasing its 
chances of success. Finally, a project’s 
output needs to be incorporated into 
production, then into market, or another 
environment than for which it was initially 
conceptualized. Additionally, one cannot 
forget about a product’s maintenance tasks 
throughout its life cycle. The R&D process 
calls for a method that can easily 
incorporate its stochastic character, and 
DES provides that capability. The ability to 
mimic the logic of the organization, and 
evaluate alternative structural alignments, 
for example, centralized versus 
decentralized, makes DES especially 
valuable. In addition, the stochastic 
character of R&D endeavors makes DES a 
very powerful tool that could be used for 
assessment of, e.g., outsourcing versus “in- 
house” R&D, and prediction of estimated 
return of investment. DES can exhibit a 
detailed level of abstraction depending on 
the modeler's need, and it is flexible and 
easy when managing different levels of 
aggregation. Moreover, many commercial 
DES packages provide techniques that can 
mimic different methods like SD and some 
even ABM. 
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SD is a method that can be very useful, 
especially at the enterprise modeling level 
when trying to capture big-picture or global 
scope of the problem. SD should be used 
where there is an existing feedback loop 
between model variables, and it is hard to 
explain the relationship by intuition itself, 
which often appears in the representation of 
R&D governance. Problems of choosing 
between various strategic options can be 
addressed with SD, as it is often hard to 
model high-level of socio-political 
phenomenon providing exact numerical 
information. This method could be 
particularly appropriate to a long-term 
model. SD structure is based on differential 
equations, but most of the commercial 
simulation packages allow for the 
incorporation of stochastic behavior into a 
model. Using this method fortesting policies 
and future trends can be invaluable when 
trying to plan in long term, which is 
specifically related to R&D governance type 
decision-making. Modeling competing 
environments allows for a comprehensive 
perspective of what is needed to obtain 
desired outcome. An important advantage 
of SD is that data used to drive a model do 
not have to be ample or even direct. Often 
one would like to learn the system and 
restore its behavior without direct 
knowledge of it, by building relations that 
are only correlated with it or related to this 
phenomenon. Another area where SD may 
be appropriate is to learn about how 
significant the roles of parts of a complex 
system are in the functioning of the whole 
body, e.g. not only how outsourcing affects 
a single project outcome, but also what are 
the long time implications. 

To manage different levels of abstraction 
one could potentially increase effectiveness 
by using nested, also known as, hybrid, 
models [14]. These models use different 
methodologies that can be arranged 
hierarchically depending on type of problem 
the model is to address. Multiple layers of 
ABM, DES, and SD methods could be 
constructed, allowing communication 
between layers and so influencing each 


other. Some structures can be also 
embedded by another if that reflects actual 
system structure. M&S practitioners often 
combine ABM and SD. These two methods 
can work well together and provide good 
combination of, e.g. lower level ABM with 
SD capturing a larger view of the system. 
Alignment of ABM-SD can be multi-layered, 
for instance, some part of agent behavior 
(internals of agent) can be described using 
differential equations from SD, then the 
environment level can be described in ABM 
terms with agents at different level of 
aggregation, and on top of that SD world- 
view, capturing global dependencies. Some 
of commercial as well as free software 
packages allow for that kind of configuration 
e.g. ANYLOGIC, and NETLOGO. In Figure 
1 , the authors propose an example of how a 
configuration of a hybrid R&D governance 
model could be assembled. 



Figure 1. One possible configuration of a 
hybrid model of R&D governance 


5.0 MODEL EXAMPLE 


The estimated number of illegal immigrants 
ranged from 8.5 to 11.8 million people 
between 2000 to 2009 [15]. While this can 
be an under- or over-estimation, many 
unofficial sources estimate it as high as over 
20 million people, which if true could cause 
a significant problem in calculations of many 
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statistics and biasing the true picture of the 
U.S. border security problem. 

Figure 2 presents a simplified conceptual 
example which captures relationships 
between two competing environments, 
(represented as capabilities: Department of 
Homeland Security (DHS) capabilities and 
criminal force capabilities), in relation to 
U.S. border security. 



Figure 2. Conceptual model of border 
security 


Money is used as a main metric in this 
model, used for both funding an increase in 
DHS capabilities and to provide illegal 
money to a criminal force. A defined 
coefficient of security influences number of 
illegal immigrants entering the U.S. and the 
amount of illegal cargo. The model allows 
for estimating the number of unauthorized 
immigrants in the U.S. 

The model has the following data-driven 
variables: population growth rates, legal 
immigration rate, Gross Domestic Products 
(GPDs) of the U.S. and the world rates, and 
federal R&D funding for the U.S. Figure 3 
presents a detailed view of the model. The 
model correlates the U.S. GPD with total 
spent money on public R&D and further on 
DHS. This can allow analysis to be 
conducted on both the country, as well as 
departmental level. The U.S. GPD further 
influences the relative ratio of difference in 
living conditions between the U.S. and the 
world. The R&D portion of the model is built 
on the concept of knowledge generation by 
both environments: one being the R&D 


environment, where conducted or 
contracted-out research by DHS is 
supposed to help eliminate illegal activity 
around the U.S. border, and the second 
environment represents knowledge of the 
criminal force to continue with their illegal 
endeavor. 

There is an assumption on the time delay 
between the reaction time needed to 
discover new techniques by both 
competitors. The authors hope that there is 
an effective way of breaking this pattern, but 
for a simplification, breaking the pattern is 
not considered in this model. Future study 
of this phenomenon using agent-based 
modeling approach is foreseen, and it would 
possibly allow for assessment of how fast a 
criminal force can adjust to changes 
imposed by border security advancements 
and vice versa. An increased border 
security coefficient can decrease effective 
smuggling and illegal border crossing, which 
in turn decreases money flow used by 
criminal force. The last part of the model 
focuses on different groups of people. Here, 
the key elements for our analysis are 
represented by the number of illegal 
immigrants and the level of human 
trafficking. The basis for obtaining those 
outputs is the assumption that the majority 
of illegal immigrants are those who cross 
the borders for economic reasons. Hence, 
the authors were looking at estimating 
differences between (GPDs) as a factor of 
estimating the number of people with a 
desire to live in the U.S. Additionally, human 
trafficking is strongly correlated with illegal 
immigration [16]. This model was partially 
validated by comparing estimated data from 
[15] and model output as shown in Fig. 4. 
The model shows potential for gaining 
valuable insight into border security, 
including estimation of the funding level 
necessary to lower illegal activity related to 
the U.S. borders. 
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Figure 4. Partial validation of the model 


6.0 Conclusions 

This paper presents insights into 
governance of R&D. The authors moved 
from theoretical knowledge of R&D 
governance, through description of popular 
approaches available for M&S practitioners, 
ending up with discussion on how one could 
model R&D governance. In addition, 
authors showed an example of SD model 
that could be used by DHS or other similar 
agency to determine R&D funding. The 
constructed model allows for estimation of 
the projected number of unauthorized 
immigrants given a current or augmented 
rate of R&D funding. Future work is 
foreseen that would incorporate hybrid 
modeling to better capture the complexities 
of R&D governance. 
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